Electronic Device and Method for Reproducing a Human Perceptual Signal

ABSTRACT

The method of reproducing a human perceptual signal comprises the steps of selecting ( 7 ) a human perceptual signal in dependency of a pace of a person ( 3 ), adapting ( 8 ) the human perceptual signal in dependency of a physical state of the person ( 3 ), and reproducing ( 9 ) the human perceptual signal. An adaptation of the human perceptual signal is limited to a certain maximum. The electronic device of the invention comprises electronic circuitry operative to perform the method of the invention. The software of the invention enables a programmable device to perform the method of the invention.

The invention relates to an electronic device for reproducing a humanperceptual signal, a music player for example.

The invention further relates to a method of reproducing a humanperceptual signal.

The invention also relates to software enabling a programmable device toperform a method of reproducing a human perceptual signal.

An electronic device for reproducing a human perceptual signal is knownfrom WO 2004/072767. WO 2004/072767 describes an audio reproductiondevice which adapts the tempo of an audio signal such that the tempo ofthe adapted audio signal lies within a predefined accepted deviationfrom a selected tempo. The selected tempo can be based on a heart rateof a person, a pace of a person and/or a user-specified tempo. Adrawback of this audio reproduction device is that it does not invoke anoptimal performance from the person, the optimal performance being theperformance that benefits the person's health the most.

It is a first object of the invention to provide an electronic device ofthe kind described in the opening paragraph, which invokes a morebeneficial performance from a person.

It is a second object of the invention to provide a method of the kinddescribed in the opening paragraph, which invokes a more beneficialperformance from a person.

The first object is according to the invention realized in that theelectronic device comprises electronic circuitry operative to select ahuman perceptual signal in dependency of a pace of a person, adapt thehuman perceptual signal in dependency of a physical state of the person,an adaptation being limited to a certain maximum, and reproduce thehuman perceptual signal. Tests have shown that simply synchronizing anaudio signal with a target pace or target physical state does notmotivate or relax the user to reach that target. To make a person changeits pace (and thus its physical state), it is important to select anaudio signal that is almost synchronized with the person's pace. Whetherthe audio signal needs to be adapted depends on whether the person needsto change its pace. Whether the person needs to change its pace dependson whether the physical state target (e.g. heart rate target) isachieved. Achieving and maintaining the physical target is beneficialfor the person's health.

The human perceptual signal may comprise audio, video, light and/ortactile feedback, for example. The electronic device may be a stationaryor a portable device. The electronic device may be used in health clubs,at home and/or outdoors, for example. The step of selecting a humanperceptual signal may comprise selecting a new content item or adaptinga human perceptual signal of an already playing content item.Preferably, the new content item is selected based on its tempo when itis a music item. Preferably, the human perceptual signal's tempo isadapted when it is an audio signal. The person's pace may be, forexample, rotations per minute while bicycling or steps per minute whilerunning. The person's physical state may be determined by measuring hisheart rate, for example. If the physical state matches a target state,the human perceptual signal need not be adapted.

In an embodiment of the electronic device of the invention, theelectronic circuitry is operative to adapt the human perceptual signalby increasing its tempo if the physical state of the person does notmeet a certain criterion. This embodiment can be used to motivate theperson to meets its physical state target, e.g. if the pace of theperson is too slow to achieve his heart rate target.

The electronic device may further comprise a user control for pausing acurrent exercise and the electronic circuitry may be configured not toincrease the tempo of the human perceptual signal while the currentexercise is paused. This prevents the electronic device from tying tomotivate the person while the person really does not want to bemotivated. A special human perceptual signal may be selected while thecurrent exercise is paused. If the target of the current exerciseincludes a target length of the exercise, the elapsed time may also bepaused.

The electronic circuitry may be operative to adapt the human perceptualsignal by decreasing its tempo if the physical state of the personexceeds a certain criterion. This can be used to relax the person tomeet its physical state target, e.g. if his heart rate is too high.

The electronic circuitry may be operative to select a human perceptualsignal in dependency of a pace of the person and the electroniccircuitry maybe configured not to adapt the human perceptual signal ifthe physical state of the person meets a certain criterion and does notexceed the certain criterion. This can be used to make the personmaintain his current pace if his physical state target is met.

The electronic device may be operative to select a new content item tobe reproduced based on a predicted pace and/or predicted physical stateof the person in a next plurality of seconds. This avoids having toselect new content items too often and to avoid having to make majoradaptations to the audio signal too often. The pace and/or physicalstate may be predicted based on the target pace and/or physical state orbased on a history of the person's pace and/or physical state.

The electronic device may further comprise a user control for skippingto a next physical exercise, the next physical exercise being associatedwith a next criterion. Each exercise may have its own physical state(e.g. heart rate) target. If the person wants to end an exerciseprematurely, this user control ensures that the electronic device doesnot motivate or relax the person incorrectly.

The second object is according to the invention realized in that themethod comprises the steps of selecting a human perceptual signal independency of a pace of a person, adapting the human perceptual signalin dependency of a physical state of the person, an adaptation beinglimited to a certain maximum, and reproducing the human perceptualsignal.

These and other aspects of the electronic device and method of theinvention will be further elucidated and described with reference to thedrawings, in which:

FIG. 1 shows an example of workout scheme that is used in an embodimentof the method of the invention;

FIG. 2 is a flow chart of an embodiment of the method of the invention;and

FIG. 3 is a block diagram of the electronic device of the invention.

Corresponding elements within the drawings are identified by the samereference numeral.

The use of music appeals to physical sport exercising, especially tosports that are individual, repetitive, and endurance. Listening tomusic, before and during exercising, is deemed to increase work outputby dissociation, by motivation, and by improving affective states and toprovide improved control and skill learning for motor control. Hence,music can be an excellent guide for keeping up to a physical exertion,especially when this exertion is monotone, repetitive, and boring.Runners with music on their ears are already an everyday appearance onthe streets.

Many recreational exercisers and endurance athletes/sportsmen work withpersonal training programs for their health and sports experience. Thesetraining programs basically define the sequence of exercises that needsto be done, and the duration and intensity of each individual exercise.They fit in a complete plan and are adapted over time, either to worktowards peak performance, weight control, or injury recovery.

The method of the invention is preferably used in a personalized musicsystem for sport exercising that realizes an integration of sporttraining programs, music selection, and music playback. It should makesport exercising more enjoyable and should help the user in keeping upwith the intended goals as set in the training program. In short, beforethe work out, the system supports the user in pre-selecting preferredmusic that suits the physical exertion in the training program. Duringthe work out, songs are selected for playback and properties of themusic playback, in particularly, the musical tempo, are adapted eitherto reflect or to guide current user performance expressed in heart rateand movement frequency. After the work out, performance and playbackdata are collected for visual inspection, for training programadaptation, and for refining future music selections by learning.

Many individual endurance sport practices require the motivation topersevere, even when these exertions evoke feelings of monotony,boredom, pain, fatigue, and exhaustion without the provision of instantrewards. In the field of Sport and Exercise Psychology, it is postulatedthat exercisers employ various associative/internal anddissociative/external strategies to cope with the physical demands dueto the physical intensity and required endurance of their sportsexertions. Under low to moderate physical load, exercisers are able tovoluntary divert attention (i.e., ‘dissociate’) from internal sensationscoming from the muscles, joints, cardiovasculary and pulmonary systems.Used dissociative strategies are imagery production, positive mooddevelopment (e.g., by thinking about the nice things in life), havingthoughts about work, career, or social relationships, creatingself-efficacy, self-esteem and confidence, or doing problem solving suchas mental arithmetic. Apparently, at approximately 70% of maximal heartrate reserve (MHRR), exercisers are forced to start switching theirattention from external cues back to internal sensations of physicaldiscomfort. At a level of physical load where feelings of extremefatigue or exhaustion prevail (e.g., at 90% VO₂ MAX), exercisers are notlonger able to ‘dissociate’ themselves from the incoming painfulsensations. Instead, exercisers have to directly ‘fight’ against thepain to persevere their exertion, for which high levels of motivationand exertion tolerance are required.

Besides making physical exertion more enjoyable, music might also helpan exerciser to dissociate from exertive feelings or even be a motivatorto sustain effort. Studies have shown that up-tempo music can enhanceperformance, though only in physical tasks of low and moderate levels ofexertion (i.e., having sub-maximal effort). Doubling the pace of themusic at the apparent critical level of 70% MHRR enables exercisers topostpone their time to fatigue; it seems that the change in musicaltempo helps them in prolonging their period of using a ‘dissociative’coping strategy. If exercisers are explicitly instructed to keep pacewith the music being played, performance will enhance. In particular,endurance enhanced in a stationary constant-pace cycling task whenpedaling rate was synchronized with music. Background music that wasasynchronous to the pedaling did not reveal the same effect as opposedto the absence of music. In high intensity exertions (e.g., at 90% VO₂MAX), the exertive feelings become too overwhelming and undifferentiatedand reach levels of extreme fatigue and exhaustion. As alreadymentioned, at that level of exertion, exercisers can not longervoluntary divert their attention to external stimuli. Hence, music willnot help them in improving to sustain their effort, though music stillbrings greater enjoyment to the task than no music.

The happy marriage between solo endurance sports and music playback hasalready resulted in consumer products. A business collaboration betweenthe electronics firm Philips and sportswear manufacturer Nike created acombined device, called MP3Run (PSA260), which acts as a portable musicplayer and a performance/training display and logging system. The playeris a 256 MB flash-memory device for playing about four to eight hoursMP3 or WMA-encoded music and contains FM radio. Music can be refreshedby down-loading. For the performance logging, it comes with a pedometerthat wirelessly transmits (using Bluetooth) speed and distance data tothe device for display and on-demand audio feedback. Performance datacan be uploaded to a personal computer for inspection. However, musicplayback or refreshing is not coupled to the performance.

A training program prescribes the sequence of physical exercises thatare scheduled in a single training. The definition of an exercise mayinclude (see FIG. 1):

The type of exercise, which refers to an exertion within sports likerunning, cycling, rowing, aerobics, and fitness;

The exercise duration, which specifies the period of time that theexercise should take, possibly at a prescribed intensity level;

The exercise intensity, which specifies at what level of exertion theexercise should be performed. It is expressed in percentage of maximalheart rate reserve (MHRR), which may specify a desired heart rate zone.For instance, the aerobic heart rate zone for improved endurance andoptimal cardiovascular training is at 70-80% of MHRR; The exercisestringency, which tells how strictly or how loosely the exercise shouldbe performed in terms of duration and intensity.

For particular exercise types, parameters are irrelevant or can not bepredicted. For instance, the heart rate during a warm up or stretchexercise is less relevant, and presumably at rest. How long it takes torecover from an exertion until the heart rate is at rest is also notknown beforehand.

Note that a training program can also consist of a single exercise suchas a run of one hour, for those who prefer ‘a loose run across the park’instead of committing themselves to training programs. Typically, thetraining program is part of a complete and personalized coaching plan toachieve a well-defined long term goal such as working towards peakperformance at a sports event, extending endurance, building up strengthand power, losing weight, getting healthier, staying healthy, orrehabilitating physically. The plans, together with the trainingprograms, are created by sports physiologists, coaches, trainers,consultancy firms for sports and health, or by the people themselves,possible helped by present-day computer technologies.

In the embodiment of the invention that is shown in FIG. 2, helping theuser in keeping up with the intended goals as set in the trainingprogram by the use of preferred music is done in four usage stages. Thefour stages are an installation stage (not shown), a preparation stage1, an exercising stage 2, and a feedback stage 4. The installationstage, the preparation stage 1, and the feedback stage 4 are off-line.In other words, the tasks involved in these stages will be performed athome using a personal computer before or after the physical exertion. Incontrast, the exercising stage 2 is on-line, while the user isphysically exercising.

Installation Stage

First-time use of the system requires an installation stage (not shown)to personalize the system. In an dialogue, the system requests relevantpersonal data (e.g., name, gender, age), exercise and physiological data(e.g., weight, fat percentage, resting heart rate, aximal or peak heartrate, stride frequency, stride length), and the level of sportsexperience (i.e., beginner, intermediate, experienced) to the user. Inaddition, the definition of exercises can be altered or added, togetherwith the likes and dislikes of music. Multiple users can use the system,though they all have their individual profile. Users can always comeback to this installation stage to reset their data.

Preparation Stage 1

The preparation stage 1 is concerned with off-line training programcreation and music pre-selection. Ready-made training programs areavailable for use, adaptation, or can be created from scratch.Subsequently, preferred music is selected from a larger music collectionthat fits the training program. Music pre-selection before the actualtraining commences ensures a hassle-free sports and music experience. Inaddition, it copes with storage limitations on present-day portableflash-based players. Different training exercises have quite differentcharacteristics in motion, pace, intensity, and duration. For instance,a warming-up is different from an interval training exercise, for itdoes not consider the exercise intensity or user performance. Instead, awarming-up requires full concentration of the exerciser. These differentexercise characteristics require different selections of preferredmusic. Both the music and the training program are downloaded to aportable player to take along. The following requirements were designedinto the music pre-selection strategy of the system:

Personal music preferences regarding likes and dislikes of songs, musicgenres, artists and the like in relation to the exercises in thetraining program are taken into account.

An amount of music is selected that is proportional to the duration ofthe individual exercises and of the training program as a whole.

Music is selected on the basis of the intensity of the exercises. Inparticular, exercise intensities are translated into musical tempoproperties.

User performance indicators are used to learn what songs need to beincluded or excluded in future pre-selection.

To formalize the music pre-selection process, we define a song as afinite vector s=(v₁, . . . v_(K)) of K attribute values. Theseattributes reflect tags of catalogue data or musically intrinsicfeatures such as unique identification, title information, artist name,genre, time duration, music tempo, and time signature. Values for thesetags can be easily provided by online services from third parties ormusic signal processing algorithms. All possible values for each k-thattribute are collected in an attribute domain denoted by D_(k).

A finite set of m songs M=(s₁, . . . s_(m)) represents the musiccollection of the user. Music pre-selection is achieved by generating aplay set that is defined as an (unordered) set of n songs p=(p₁, . . .p_(K)) where p_(i)∈M for i=1, . . . , n. We use p_(ik) to denote thevalue of the k-th attribute of the i-th song in the play set. Fornotational convenience, we consider the size of the play set n to beknown. In practice, the size of the play set is made variable.

For each individual exercise, a set of constraints C is defined, eitherby the user or by the system from which a play set can be generated. Inaddition, constraints are defined that should hold for the trainingprogramme as a whole, for instance, to guarantee that different songsare selected for the exercises and that all music fits on the portableplayer. All play sets are combined into a collection of pre-selectedmusic to be used in the exercising stage. Constraints are restrictionsthat tell in declarative terms what songs should be in the play set andwhat songs should not. Constraints can reflect music preferences of theuser or restrictions on music attributes that suit the physical exertionin an exercise. A play set is said to satisfy a constraint if it meetsthe restrictions that are defined by that constraint. Otherwise, it issaid to violate the constraint.

If the user dislikes music with particular attributes (e.g., an artistor genre) while doing a particular exercise, she can prevent theselection of this type of music by notifying the system about this factthrough a user interface. The system then instantiates an excludeconstraint. Formally, the exclude constraint is given by a four-tuple(p, n, k, V), where:

p is a play set,

n is the size of p,

k is an attribute number, 1≦k≦K,

V is a set of attribute values, V⊂D_(k),

denoting that it has to hold that p_(ik)∉V, for all 1≦i≦n.

If, on the other hand, the user likes music with particular attributes,the system instantiates a counting constraint that restricts and forcesthe number of occurrences of particular attribute values in the playset. Formally, the counting constraint is given by a six-tuple (p, n, k,V, a, b), where:

p is a play set,

n is the size of p,

k is an attribute number, 1≦k≦K,

V is a set of attribute values, V⊂D_(k),

a and b are a lower bound and an upper bound, respectively,

denoting that it has to hold that a≦|{1≦i≦n|p_(ik)∈V}|≦b.

If an exercise has a limited duration, a set of songs needs to beselected that has a similar total duration, for which a durationconstraint can be used. Formally, the duration constraint is given by afive-tuple (p, n, k, a, b), where:

p is a play set,

n is the size of p,

k is an attribute number, 1≦k≦K,

-   -   a and b are a lower bound and an upper bound, respectively,

denoting that it has to hold that

$a \leq {\sum\limits_{i = 1}^{n}p_{ik}} \leq {b.}$

By its intensity and stringency, an exercise restricts the play setgeneration on the range of musical tempi that need to be included. If weassume that movement frequency will be synchronized with the music (orvice versa), only music with a specific range of musical tempi isrequired, for which a range constraint is available. The preciserestrictions on music tempo in relation to the exercises need to belearned for each specific user. Formally, the range constraint is givenby a five-tuple (p, n, k, v, w), where:

p is a play set,

n is the size of p,

k is an attribute number, 1≦k≦K,

v and w are a lower bound and an upper bound, respectively, where v,w∈D_(k)

denoting that it has to hold that v≦p_(ik)≦w, for 1≦i≦n. Likewise, songscan be generated for the playset as if the values for the k-th attributeare drawn from a statistical distribution (e.g., a Normal distribution).

Note that constraints that restrict the order of songs are not modelled,as play sets are considered un-ordered collections.

1) Implementation by local search: Satisfying a set of constraints is aNP-hard combinatorial problem. It is thus unlikely that a polynomialalgorithm exists that computes a playset that meets any given set ofconstraints. If constraints are conflicting, there exists even nofeasible solution. Therefore, instead of striving for an exact solution,an approximate solution is computed by using local search. In this localsearch approach, the constraints on each playset are translated intonormalized, piece-wisely linear penalty functions. Each penalty functionis defined to be zero if the constraint is met, to be larger than zeroif the constraint is not met, and to increase with the amount ofviolation of the constraint. As an example, the penalty function for theduration constraint is defined as

${f\left( {p,n,k,a,b} \right)} = \left\{ \begin{matrix}{0,} & {a \leq \sigma \leq b} \\{\frac{a - \sigma}{\delta},} & {\sigma < a} \\{\frac{\sigma - b}{\delta},} & {\sigma > b}\end{matrix} \right.$

where

$\sigma = {\sum\limits_{i = 1}^{n}p_{ik}}$

and δ=max(a−n·min D_(k), n·max D_(k)−b). The penalty is zero, if thesummation of all k-th attribute values (e.g., total duration) is withinthe range [a, b]. Otherwise, the penalty is a normalized differencebetween that summation and the closest lower or upper bound.

The total penalty is defined as a weighted convex combination of allpenalty functions involved. Weights indicate how severe it is if aconstraint has not been met. Some constraints are very hard, whereasother constraints indicate wishes that might be partly violated.Selecting too few songs of a preferred artist may be more severe thanselecting more music than that can fit in an exercise.

To find a playset that minimizes the total penalty, local searchconsiders complete solutions or playsets. It steps from solution tosolution by applying a random, small change at each iteration. The kindof changes to be allowed are: (i) adding a song, (ii) deleting a song,and (iii) replacing a song by a new one. Changing the order of songs isless relevant as constraints on sequence order are not addressed here.If the newly generated music selection is better than the original one,the new one is accepted and we enter the next iteration of local search.If the new solution is worse, the solution may be accepted with acertain probability, which decreases with the amount of deterioration,but also decreases during the course of the algorithm. In other words,the search process is ‘cooling down’, known as simulated annealing, toavoid becoming trapped at local minima. Simulated annealing requires apre-defined cooling schedule for the temperature control; a standardlinear cooling schedule is used.

A small modification to the above approach is not converting allconstraints into penalty functions, but to handle a few easy constraintsby pre-processing (i.e., by using constraint propagation principles).For instance, if the user wants all songs to be rock songs, one caneasily achieve that by only using the rock songs from the musiccollection, and leaving all other songs out.

Exercising Stage 2

The exercising stage 2, see FIG. 2, is an on-line stage in which thepre-selected music is played back while the user is exercising.Exercises in the training program dictate what music will be playedback. The system (now, a portable player) monitors and records userperformance data like heart rate using a wireless connection with aheart rate sensor, and movement frequency (i.e., stride frequency forrunners) and running speed using a wireless connection with a pedometer.Dependent on the required mode of motivation, music playback is adaptedto the current user performance or current training goal. Besidesstandard linear playback, the following advanced music playback modeshave been realized:

Constant tempo: music is played back at a constant tempo to motivate theexerciser to synchronize her steps with the music for enhancedendurance. Its aim is to help the exercise to evenly distribute hereffort over a distance and reducing recovery costs due to brief highintensity efforts.

Pace matching: music tempo is continually adapted to the current stridefrequency of the exerciser to support the running at varying stridefrequencies without having to concentrate on keeping time.

Pace influencing: music tempo is sped up/slowed down to motivate theexerciser to speed up/slow down in the same manner by synchronizing hersteps with the music. In this case, the goal is to arrive at aperformance level as defined in the training program.

The method performed in the pace influencing mode comprises a step 7 ofselecting a human perceptual signal in dependency of a pace of a person3, a step 8 of adapting the human perceptual signal in dependency of aphysical state of the person, an adaptation being limited to a certainmaximum, and a step 9 of reproducing the human perceptual signal.

The different modes of playback are relevant for different ways ofexercising that can vary in type of motion (e.g., running, cycling,interval, jogging), pace, intensity, and duration. The assumption isthat the performance of the user is measured, while exercising. The userperformance can be expressed as running pace in step frequency using apedometer, as cycling pace in revolution frequency using a cyclometer,or as heart rate in heartbeats per minute using a cardiometer. Forrunning, the step frequency can be taken as unit of performance. Forcycling, half the cycling frequency can be taken as unit of performance.For rowing, half of the full rowing cycle can be taken as unit ofperformance. For heart rate, a single heartbeat can be taken as unit ofperformance. Different units of performance can be defined for differentsports. Properties of music playback (e.g., tempo, loudness, soundposition, equalization) are then adapted on the basis of the currentuser performance and currently selected playback mode. For instance, thetempo of the music playback can be controlled by music transformationtechnologies (e.g., time stretching), if required. The modes of playbackare extended over songs, implying that new songs are chosen on the basisof the playback mode and the current user performance.

Obviously, the user has control on the playback; songs and exercises canbe skipped or prolonged using interactive controls on the player. Also,different modes of playback can be selected. Lastly, the duration andthe intensity of the current exercise are monitored for signaling theuser to go to the next exercise. Song selection is based on the currentuser performance, current exercise goal, mode of music playback, andhistory of already played songs.

The different modes of music playback require different control andmusic selection strategies. Standard linear playback requires no furtherelaboration. The system is set-up with a portable music player, heartrate monitoring (typically comprising of a chest belt with a wirelesstransmitter, and a receiver connected to the player) and stridefrequency monitor (using a pedometer connected to the player). The usedchest belt transmits a pulse for every heart beat. The system uses afive-point moving average of incoming pulse intervals as the currentheart rate. For stride frequency, a five-point moving average is used aswell.

A) Constant Tempo: Music playback by constant tempo transforms the tempoof a given song by time stretching, if the original tempo of the songdoes not meet the desired tempo. In short, time stretching is theshortening or lengthening of a audio data file without pitchmodifications according to a scaling factor. Obviously, this needs tohappen faster than real-time for online stretching and playback; astandard time-domain technique based on synchronous overlap-and-add(SOLA) is used. The scaling factor represents the ratio between thedesired tempo and the original tempo of the song. Time stretching cannot be done with impunity for extreme scale factors, as music that ismodified too extensively (e.g., more than 125%) sounds awkward.Therefore, the (next) song for playback is defined as the one whoseoriginal tempo comes closest to the desired tempo and which has not beenplayed back recently. If desired, songs by one artist are not playedsuccessively.B) Pace matching: Exercisers are motivated by music that is synchronizedto their movements. Pace matching synchronizes the music to the user'sstride frequency. Stride frequency, or pace, is the number of groundimpacts of the user's feet in the course of one minute and is denoted insteps per minute (spm). The music tempo is denoted in beats per minute(bpm) where one musical measure in a meter of 4/4 consists of 4 beats.In order to match the music tempo to the stride frequency, the musicneeds to be played faster or slower by using time stretching. Typicalmaximal values for the scaling factors involved in time stretching are−15% to +25%. For a song with a tempo of 100 bpm, this results in astretch range of [85, 125].

Playing music in a tempo of 75 bpm when the stride frequency is 150 spmis also considered a match, as the moment of a foot impact can stillcoincide with a musical beat. Note that we assume that the music tempoand the user's strides are in phase. In general, multiples (2×, 4×, 8×,. . . ) or integral divisions

$\left( {\frac{1}{2},\frac{1}{4},\frac{1}{8},\ldots}\mspace{11mu} \right)$

of stride frequency are considered matching with the music tempo.

For instance, if the tempo of the current song is 100 bpm (meaning asong stretch range of [85, 125]) and the stride frequency is 210 spm,the system would change the music playout tempo to 105 bpm. If thestride frequency is 130 spm, no valid match inside the song stretchrange can be found. In this case, the music playout tempo is changed tothe stretch border closest to the stride frequency by time stretchingthe song. Then, a new song is selected, under the condition that thecurrent song has been played long enough (say, 30 seconds, to reducerepetitive song changing over time). The new song is required to have astretch range that includes the current playout tempo as well as thecurrent stride frequency, in order to allow for a transition withoutabrupt tempo changes.

Changing playout tempo is done in a smooth fashion. When a change intempo is done instantly, a hick-up in the music playback is evident. Onthe other hand, when the change takes a long time, system response istoo slow (i.e., in excess of 10 seconds) to accurately match the user'sstride frequency. The parameter that specifies system response time inpace matching mode is T_(m), which specifies the time that a maximalchange in playout tempo as defined by the song's stretch range shouldtake. A linear function is used to calculate the actual time to changefrom a given playout tempo to a new playout tempo. This function,denoted by t_(m), is supplied with two points, t_(m)(0)=0 andt_(m)(Δ_(max))=T_(m), where Δ_(max) denotes maximal change in playouttempo. A linear relationship is assumed, but as a future extension, asquare root relationship can be used to propagate small changesrelatively faster than larger ones.

When considering the above example with an assumed T_(m)=1000 msecs, achange from 100 to 105 bpm would take

${t_{m}\left( \Delta_{100->105} \right)} = {\frac{{100 - 105}}{{85 - 125}} = {125\mspace{11mu} m\; {{secs}.}}}$

C) Pace Influencing: To motivate exercisers to reach and to keep theirheart rate within a certain heart rate zone, the system uses a four-stepmethod:1) Match the music playout tempo with the user's stride frequency toensure the connection between them;2) Determine heart rate goal and, from that, stride frequency and musicplayout tempo subgoals;3) Propagate the change in music playout tempo from the current playouttempo to the playout tempo desired by the determined subgoal.4) Wait for heart rate stabilization.

These steps are repeated in a continuous loop until the exercise goalhas been reached.

The assumption is that by matching music playout tempo to stridefrequency in step 1, a change in playout tempo in step 3 will have asynchronization effect on the user's stride frequency. In other words,the exerciser will automatically keep her movements in time with laterchanges in music playout tempo. Matching playout tempo and stridefrequency is done by using the pace matching mode described earlier. Assoon as the stride frequency is matched, the system advances to the nextstep.

In the second step, the user's current heart rate is compared with theheart rate goal of the current exercise, and the difference percentageis determined. A prediction of the required stride frequency is made byapplying the heart rate difference percentage directly to the currentstride frequency. From sports physiology, we know that heart rate willalter linearly with exercise intensity. In other words, an increase of10% in heart rate means a 10% increase in stride frequency. The desiredstride frequency is then transformed into a desired music playout tempo;the desired playout tempo can be a multiple or integral division of thedesired stride frequency.

Like in pace matching mode, when the current song can not be furtherstretched to accommodate the change in music playout tempo, the systemchanges the playout tempo to the closest value possible (i.e., eitherthe maximal or minimum song stretch boundary). Subsequently, it selectsa new song that has a stretch range that includes the reached and thedesired music playout tempo.

The system keeps a history of heart rate measurements. From that, it canmake a prediction how the heart rate (and hence the desired playouttempi) will progress in the next 30 seconds. This prediction providesvaluable information on what song is best to choose next, if songchanging is required for playback. It effectively minimizes the numberof song changes needed.

A change in music playout tempo is not carried out immediately, since anabrupt change in playout tempo would cause the user not to follow thechange, either because of ignorance or fatigue. Instead, the systemtakes into account a propagation time. Like in pace matching mode, alinear function is used to calculate the actual time to change from agiven playout tempo to a new playout tempo. This function, denoted byt_(i) is supplied with two points, t_(i)(0)=0 and t_(i)(Δ_(max))=T_(i),where Δ_(max) denotes maximal change in playout tempo.

Finally, when the change has been propagated, the system waits for theuser's heart rate to stabilize before making additional changes. Fromexperiences when testing, the user's cardiovascular system needs sometime to adapt to the additional effort. Rather than specifying a fixedwaiting time after each change, the system looks for a trend ofdecreasing differences in successive heart rates with small derivativevalues.

In addition, to encourage users in working out more intensively, themusic can be played louder, lower frequencies can be accentuated, and/orthe music can be played back as if it is in front of the listener. Newsongs can be selected that have a tempo that corresponds to a level ofperformance that is slightly higher than the current user performance.In addition, to encourage users in working out less intensively, themusic can be played softer, lower frequencies can be attenuated, and/orthe music can be played back as if it is behind the listener. New songscan be selected that have a tempo that corresponds to a level ofperformance that is slightly lower than the current user performance.

Feedback Stage 4

The feedback stage 4 is concerned with off-line performance storage andfeedback. Performance data, the training program actually performed, andthe list of songs actually played back is stored for user inspection andsystem learning purposes. The performance data can be graphicallydisplayed for inspection, self-esteem, motivation, and for advising onadapting the current training method. The system acquires data forlearning what music is best suitable for what type of exercise.

Finding Music that Fits a Training Program

The most important characteristic that determines the suitability of asong in the music database for use in a particular exercise is itstempo. Other characteristics include, for example, its percussiveness(the amount of beats in the song), its meter, its key, the musicalinstruments used in the recording, etc. Since the amount of storagespace is often limited, a way has to be found to ensure the right musicis on the portable device so that a suitable song can be found at alltimes, taking into account user preferences such as ‘do not play thesame song twice during one training program’ or ‘do not play songs bythe same artist successively’. This can be achieved by generating atempo probability graph per exercise and then combining these graphs toa global tempo probability graph for the entire training program.

In an embodiment of the method of the invention, the pace that matchesthe exercise heart rate zone is determined in order to find music thatfits the training program. Unfortunately, this relationship is not astraightforward one. The heart rate resulting from running at a certainpace varies from user to user and over time due to factors such as,among others, endurance (how long is the running already going on),fatigue, fitness of the runner, the weather conditions and the time ofday. However, certain average tempos may be assumed, resulting in atable like:

HR gender age pace 50% male under 30 90.2 50% male 30-40 88.6 60% maleunder 30 110.1 70% male under 30 116.5 70% female under 30 112.9 70%female 30-40 110.7 80% male under 30 125.0

Another important aspect for determining the average tempo for anexercise is the position of the exercise within the training program.For instance, the pace needed to reach a 110 heart rate when startingexercising might be 90, while after 30 minutes a pace of 80 will beenough to reach this heart rate. Therefore, a user's fatigue curve canbe assumed based on gender and age, in which the influence of time onthe pace needed to reach a certain heart rate level is documented. Theserough tables and fatigue curves are based on classifications of usersbecause of gender and age groups. This information can be entered by theuser easily and does not require lots of time for the system'scalibration.

Having established a predicted value for the pace of each of theexercises in a training program, a Gaussian distribution can be assumedfrom these average paces, estimating the likelihood of a particulartempo in an exercise with a typical heart rate zone:

${f(x)} = {\frac{1}{\sigma \sqrt{2\pi}}^{{{- {({x - \mu})}^{2}}/2}\sigma^{2}}}$

The mean μ in this straightforward Gaussian function is the averageestimated pace for an exercise, where as the standard deviation σdenotes the width of the resulting graph. The standard deviation can bedetermined taking in consideration the estimated tempo of the previousexercise (as it is likely that this tempo is the tempo at which thecurrent exercise will be started) and the next exercise (for similarreasons). All of the above takes into account statistical information onthe performances of user classes (age and gender groups). Also,information from the training program is used. However, during exercise,data on the user's performance is logged and this is collected in thefeedback stage, after exercising. This data can be used to make thedistributions more personal.

First, the average deviation can be determined for certain types ofexercises and this can be used in conjunction with the statisticalinformation already available to find a better standard deviation σ forthis class of exercises. Also the mean tempo μ in the function can bedetermined much more precise when taking into account the user'sperformance. An effective way to perform this combination would be togenerate a real curve from the user's performance and average out thiscurve with the one already created before. The same process can beapplied to the user's fatigue curve. In this way, the prediction oftempo is getting more and more precise with each new exercise that iscompleted by the user.

Having created more or less precise distributions for each individualexercise, all these curves can be summed up to one global distributionfor the entire training program. The relative weight of the exercisescan be taken into account. This weight is determined by looking at theduration of the exercises with respect to the length of the entiretraining program. One of the problems that arises is that some exercisedurations may be expressed in kilometres or target heart rate instead oftime. For the distance duration exercises, the distance may bemultiplied with the average running speed, taking into account theuser's fatigue curve. For target heart rate, the user's fatigue curvecan be taken into account as well.

When a global distribution has been established for the entire trainingprogram, this distribution can be mapped to total playing time availableon a music (player) memory. Preferably, the total playing time isdependent on the amount of space on the memory and the averagecompression level for individual songs. Matching this graph can be madeone of the constraints that songs need to satisfy in the Preparationstage 1, see FIG. 2.

User Interface

In an embodiment of the system, the system allows a user to stop, skip,pause or resume an exercise in a training program, the currently playingsong in the playlist, and the mode of music playback. The music playerkeeps itself strictly to the training program by playing only music thatwas pre-selected for individual exercises and by taking into account theduration of the exercises or the time it takes to reach a pre-definedperformance level. User control of the music playback is effectuated bythe present invention consisting of a user interface system thatprovided access to interactive controls.

The system starts by playing music that has been pre-selected for thefirst exercise. The type of exercise is conveyed to the user, either byspeech synthesis and/or a textual/display cue. Also, song information isconveyed to the user, for instance, by display and/or speech synthesis.Exercises can have a pre-defined fixed duration or can have a variableduration. In the latter case, the exercise will be prolonged until apre-defined level of user performance has been reached or as long as apre-defined level of user performance has to be maintained (see FIG. 1).

While doing the exercise and listening to music, a user can pause themusic playback and the exercise, for instance, when full attention isrequired for other tasks such as crossing a road or talking to apasser-by. The music and the exercise can be resumed afterwards.Resumption may mean re-starting an exercise, if the exercise deals withreaching or maintaining a particular level of performance. The user isinformed about the state of the exercise, either by display and/orspeech synthesis.

If playback of a particular song is disliked, the user can skip thecurrently playing song. Then, a next pre-selected song will be playedback. Choice of this new song may depend on the mode of playback.Information about the new song is conveyed to the user. Skipping songscan be done repetitively, until a preferred song is found. The choice ofnew songs is cyclic, allowing for user error recovery and fullnavigation through all songs that are pre-selected for an exercise.

The user is informed when an exercise can be ended. Ending of anexercise means that the pre-defined duration has expired or the usersuccessfully reached or maintained a particular level of performance.This information can be conveyed by an auditory cue, a vibratory cue, atextual/display cue, a LED-light cue, or by changing properties of themusic playback (e.g., slowing down, softer). The user then decides tomove on to the next exercise by skipping the current exercise. The usermay also decide to stay with the current exercise; after a short period,the device will re-start the current exercise. At all times, the usercan decide to skip the current exercise, even if the exercise has notbeen ended yet. Moving on to the next exercise is cyclic. If one is atthe last exercise in the training program, the next exercise will be thefirst one in the program. This cyclic nature allows for user errorrecovery and full navigation. Other embodiments of the system may haveonly some of the feature described above.

The electronic device 21 of the invention comprises electronic circuitry23, see FIG. 3. The electronic circuitry 23 is operative to select ahuman perceptual signal in dependency of a pace of a person, adapt thehuman perceptual signal in dependency of a physical state of the person,an adaptation being limited to a certain maximum, and reproduce thehuman perceptual signal. The electronic device 21 may be stationary orportable. The electronic device 21 may be, for example, a contentreproduction device for reproducing the human perceptual signal whilethe person is performing the physical activity. The electronic device 21may comprise an input 25, e.g. a USB port or a network connection, forreceiving content items. The electronic device 21 may comprise an output25, e.g. a USB port, a network connection, or an earphones jack, forreproducing the human perceptual signal on another device. Theelectronic device 21 may comprise a (fixed or removable) storage means31, e.g. a hard disk, solid-state memory and/or an optical disc reader.The storage means 31 may store content items, for example. Theelectronic device 21 may comprise a reproduction means 29, e.g. aspeaker, a tactile feedback generator, a display or a lamp, forreproducing the human perceptual signal. The electronic circuitry 23 maybe, for example, a general-purpose or application-specific processor.Software can be used to make the electronic circuitry 23 operative toperform the method of the invention.

While the invention has been described in connection with preferredembodiments, it will be understood that modifications thereof within theprinciples outlined above will be evident to those skilled in the art,and thus the invention is not limited to the preferred embodiments butis intended to encompass such modifications. The invention resides ineach and every novel characteristic feature and each and everycombination of characteristic features. Reference numerals in the claimsdo not limit their protective scope. Use of the verb “to comprise” andits conjugations does not exclude the presence of elements other thanthose stated in the claims. Use of the article “a” or “an” preceding anelement does not exclude the presence of a plurality of such elements.

‘Means’, as will be apparent to a person skilled in the art, are meantto include any hardware (such as separate or integrated circuits orelectronic elements) or software (such as programs or parts of programs)which perform in operation or are designed to perform a specifiedfunction, be it solely or in conjunction with other functions, be it inisolation or in co-operation with other elements. The invention can beimplemented by means of hardware comprising several distinct elements,and by means of a suitably programmed computer. ‘Software’ is to beunderstood to mean any software product stored on a computer-readablemedium, such as a floppy disk, downloadable via a network, such as theInternet, or marketable in any other manner.

1. An electronic device (21) comprising electronic circuitry (23), theelectronic circuitry (23) being operative to: select (7) a humanperceptual signal in dependency of a pace of a person; adapt (8) thehuman perceptual signal in dependency of a physical state of the person,an adaptation being limited to a certain maximum; and reproduce (9) thehuman perceptual signal.
 2. An electronic device (21) as claimed inclaim 1, wherein the electronic circuitry (23) is operative to adapt (8)the human perceptual signal by increasing its tempo if the physicalstate of the person does not meet a certain criterion.
 3. An electronicdevice (21) as claimed in claim 2, further comprising a user control forpausing a current exercise and wherein the electronic circuitry (23)does not increase the tempo of the human perceptual signal while thecurrent exercise is paused.
 4. An electronic device (21) as claimed inclaim 1, wherein the electronic circuitry (23) is operative to adapt (8)the human perceptual signal by decreasing its tempo if the physicalstate of the person exceeds a certain criterion.
 5. An electronic device(21) as claimed in claim 1, wherein the electronic circuitry (23) isoperative to select (7) a human perceptual signal in dependency of apace of the person and the electronic circuitry does not adapt (8) thehuman perceptual signal if the physical state of the person meets acertain criterion and does not exceed the certain criterion.
 6. Anelectronic device (21) as claimed in claim 1, wherein the electronicdevice (23) is operative to select a new content item to be reproducedbased on a predicted pace and/or predicted physical state of the personin a next plurality of seconds.
 7. An electronic device (21) as claimedin claim 1, further comprising a user control for skipping to a nextphysical exercise, the next physical exercise being associated with anext criterion.
 8. A method of reproducing a human perceptual signal,comprising the steps of: selecting (7) a human perceptual signal independency of a pace of a person (3); adapting (8) the human perceptualsignal in dependency of a physical state of the person (3), anadaptation being limited to a certain maximum; and reproducing (9) thehuman perceptual signal.
 9. Software enabling a programmable device toperform the method of claim 8.